Skip to content

Refactor/lexer hex optimization ,#5108#5109

Open
MayankRaj435 wants to merge 1 commit intoboa-dev:mainfrom
MayankRaj435:refactor/lexer-hex-optimization
Open

Refactor/lexer hex optimization ,#5108#5109
MayankRaj435 wants to merge 1 commit intoboa-dev:mainfrom
MayankRaj435:refactor/lexer-hex-optimization

Conversation

@MayankRaj435
Copy link
Contributor

@MayankRaj435 MayankRaj435 commented Mar 16, 2026 •
edited
Loading

Optimize hex escape parsing in lexer

This Pull Request fixes/closes #5108.

It changes the following:

  • Performance Optimization: Refactored take_unicode_escape_sequence and take_hex_escape_sequence in core/parser/src/lexer/string.rs to parse hexadecimal digits directly from the character stream into a u32 accumulator.
  • Allocation Removal: Eliminated unnecessary heap allocations (Vec<u32> and String) that were previously used as buffers for u32::from_str_radix and u16::from_str_radix.
  • Code Cleanup: Removed the take_until method from core/parser/src/lexer/cursor.rs as it is no longer used by the optimized escape sequence parsing.
  • Lint Fixes: Cleaned up unused imports in core/parser/src/lexer/cursor.rs discovered after the refactoring.

Verification Results:

  • Lexer Tests: All 63 tests in core/parser/src/lexer/tests.rs passed successfully.
  • Parser Tests: Verified broader compatibility by running the full boa_parser test suite.
  • Style/Linting: Passed both cargo fmt and cargo clippy -- -D warnings without any issues.

@MayankRaj435 MayankRaj435 requested a review from a team as a code owner March 16, 2026 07:10
@github-actions github-actions bot added the Waiting On Review Waiting on reviews from the maintainers label Mar 16, 2026
@github-actions github-actions bot added this to the v1.0.0 milestone Mar 16, 2026
@github-actions github-actions bot added C-Parser Issues surrounding the parser C-Builtins PRs and Issues related to builtins/intrinsics labels Mar 16, 2026
@MayankRaj435
Copy link
Contributor Author

MayankRaj435 commented Mar 16, 2026

@jedel1043 @nekevss Please review it once at your convenience ,thanks

@github-actions
Copy link

github-actions bot commented Mar 16, 2026 •
edited
Loading

Test262 conformance changes

Test result main count PR count difference
Total 52,963 52,963 0
Passed 50,073 50,073 0
Ignored 2,072 2,072 0
Failed 818 818 0
Panics 0 0 0
Conformance 94.54% 94.54% 0.00%

Tested main commit: daf86d30277a5c1fb353825347f75c1d83516905
Tested PR commit: 82dece19c7b7132d5e7ab480ddf3d1e6d59e301e
Compare commits: daf86d3...82dece1

@codecov
Copy link

codecov bot commented Mar 16, 2026 •
edited
Loading

Codecov Report

❌ Patch coverage is 72.97297% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.30%. Comparing base (6ddc2b4) to head (82dece1).
⚠️ Report is 885 commits behind head on main.

Files with missing lines Patch % Lines
core/parser/src/lexer/string.rs 72.97% 10 Missing ⚠️
Additional details and impacted files 
@@             Coverage Diff             @@
##             main    #5109       +/-   ##
===========================================
+ Coverage   47.24%   59.30%   +12.06%     
===========================================
  Files         476      580      +104     
  Lines       46892    63127    +16235     
===========================================
+ Hits        22154    37438    +15284     
- Misses      24738    25689      +951

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

jedel1043
jedel1043 reviewed Mar 16, 2026
View reviewed changes
core/parser/src/error/mod.rs
Copy link
Member

@jedel1043 jedel1043 Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated change from another PR

Copy link
Contributor Author

@MayankRaj435 MayankRaj435 Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed those ,please review it .Thanks

@jedel1043 jedel1043 added Waiting On Author Waiting on PR changes from the author and removed Waiting On Review Waiting on reviews from the maintainers labels Mar 16, 2026
@MayankRaj435 MayankRaj435 force-pushed the refactor/lexer-hex-optimization branch from 487893d to 0933a1e Compare March 17, 2026 03:19
@github-actions github-actions bot added the Waiting On Review Waiting on reviews from the maintainers label Mar 17, 2026
@jedel1043 jedel1043 removed the Waiting On Review Waiting on reviews from the maintainers label Mar 17, 2026
@MayankRaj435 MayankRaj435 force-pushed the refactor/lexer-hex-optimization branch from 012b02d to 82dece1 Compare March 18, 2026 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jedel1043 jedel1043 jedel1043 left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

C-Builtins PRs and Issues related to builtins/intrinsics C-Parser Issues surrounding the parser Waiting On Author Waiting on PR changes from the author

Projects

None yet

Milestone

v1.0.0

Development

Successfully merging this pull request may close these issues.

Optimize the parsing of Unicode and Hexadecimal escape sequences in the lexer to avoid unnecessary heap allocations

2 participants

@MayankRaj435 @jedel1043